SGEM: stochastic gradient with energy and momentum

In this paper, we propose SGEM, stochastic gradient with energy and momentum, to solve a class of general non-convex optimization problems, based on the AEGD method introduced in (adaptive descent energy) Liu Tian (Numerical Algebra, Control Optimization, 2023). SGEM incorporates both momentum so as inherit their dual advantages. We show that features an unconditional stability property provide positive lower threshold for variable. further derive energy-dependent convergence rates setting, well regret bound online convex setting. Our experimental results converges faster than generalizes better or at least SGDM training some deep neural networks.